Minimally invasive user metadata

ABSTRACT

A method of dynamic tagging of image objects appearing in media content items associated with a user. The method comprises automatically identifying a plurality of media objects by an image processing analysis of a plurality of media content items of a user, analyzing an output of at least one sensor of a client device for identifying a plurality of suitable push GUI events for presenting a graphical user interface (GUI) to a user of said client device, in each one of said plurality of suitable push GUI events: triggering a presentation of one of said plurality of media objects using said GUI on a display of said client device, said GUI encourages said user to input user defined metadata describing said presented media object, acquiring said user defined metadata from said user using said GUI, and documenting said user defined metadata in association with said presented media object.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/818,825 filed Nov. 21, 2017, which is a continuation of U.S. patentapplication Ser. No. 15/044,128 filed on Feb. 16, 2016, now U.S. Pat.No. 9,830,055.

The contents of the above applications are incorporated herein byreference in their entirety.

FIELD AND BACKGROUND OF THE INVENTION

The present invention, in some embodiments thereof, relates to imageobject tagging and, more specifically, but not exclusively, to methodsand systems of tagging image objects in images of a specific user.

During the last years, since the advent of digital photography and inparticular after the thriving social networks, there are a flood ofimages and videos which are uploaded to network accessible documents andare uploaded to user computers and data storage, such as webpages andshared folders.

SUMMARY OF THE INVENTION

According to some embodiments of the present invention, there isprovided a method of dynamic tagging of image objects appearing in mediacontent items associated with a user, comprising: automaticallyidentifying a plurality of media objects by processing of a plurality ofmedia content items of a user, identifying a plurality of suitable pushgraphical user interface (GUI) events for presenting a GUI to a user ofthe client device, in each one of the plurality of suitable push GUIevents: triggering a presentation of at least one of the plurality ofmedia objects using the GUI on a display of the client device, the GUIencourages the user to input user defined metadata describing thepresented media object, acquiring the user defined metadata from a userinput of the user using the GUI, and documenting the user definedmetadata in association with the presented media object.

Optionally, the automatically identifying a plurality of media objectsis performed by image analysis.

More optionally, the identifying a plurality of suitable push GUI eventsfurther comprises: sequentially identifying the plurality of suitablepush GUI events by a behavior analysis a user based on a plurality ofoutputs of at least one sensor of the client device, and identifying theplurality of suitable push GUI events according to the behavioranalysis.

More optionally, the analyzing data executed on a client device furthercomprising: sequentially identifying the plurality of suitable push GUIevents by an analysis of execution of a plurality of applications by ahardware processor of the client device, calculating a usage patternaccording to the analysis, and identifying the plurality of suitablepush GUI events according to the usage pattern.

Optionally, the plurality of media objects comprises at least one memberof a group consisting of an unhuman animal, an art piece, a roombackground, a furniture, and a car.

Optionally, the automatically identifying comprises applying at leastone image recognition process on each of the plurality of media contentitems to identify a predefined media object therein.

Optionally, the plurality of media objects comprises a plurality ofimages captured using an image sensor of the client device and stored ina media library of the client device.

Optionally, the plurality of media objects comprises a plurality ofimages captured using an image sensor of another device and stored in astorage outside of the client device.

Optionally, the method comprises receiving a search query comprising atleast one keyword matching to respective the user defined metadata fromthe user, matching the at least one keyword to a database associatingthe user defined metadata of each of the plurality of media objects to aplurality of additional descriptive keywords, and using the outcome ofthe matching as an input to a media content object search engine.

Optionally, the plurality of suitable push GUI events are timed spacedfrom one another by a period of at least one day.

Optionally, the GUI is a window presenting at least a portion of one ofthe plurality of media content items that depicts one of the pluralityof media objects together with a question encouraging a textual input ofthe input user defined metadata describing the presented media object.

Optionally, the GUI is a window presenting at least a portion of one ofthe plurality of media content items that depicts a marked version ofone of the plurality of media objects.

More optionally, wherein GUI comprises a marking tool allowing a user tomark an additional media object on the portion for providing additionaluser defined metadata.

More optionally, further comprising identifying the presented mediaobject in at least one new media content item and automaticallyassociating the at least one new media content with respective the userdefined metadata.

Optionally, the method comprises receiving a search query comprising atleast one keyword matching to respective the user defined metadata fromthe user and retrieving a respective content media item depictingrespective the media object in response to the search query.

According to some embodiments of the present invention, there isprovided a method of dynamic tagging of image objects appearing in mediacontent items associated with a user. The method comprises tagging amedia content item of a user with user defined metadata describing amedia object appearing in the media content item, the user definedmetadata is acquired using a GUI presented on a display of a userterminal of the user, identifying a presence of the media object in aplurality of new media content items by an image processing analysis,and tagging each of the plurality of new media content items with theuser defined metadata.

According to some embodiments of the present invention, there isprovided a client device of dynamic tagging of image objects appearingin media content items associated with a user, comprising: a databasecomprising a plurality of media content items of a user, a program storestoring code, and a processor coupled to the program store for executingthe stored code, the code comprising: code instructions forautomatically identifying a plurality of media objects by processing ofthe plurality of media content items, code instructions for identifyinga plurality of suitable push GUI events for presenting a graphical userinterface (GUI) to a user of the client device, code instructions forperforming the following in each one of the plurality of suitable pushGUI events: triggering a presentation of one of the plurality of mediaobjects using the GUI on a display of the client device, the GUIencourages the user to input user defined metadata describing thepresented media object, acquiring the user defined metadata from theuser using the GUI, and documenting the user defined metadata inassociation with the presented media object.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIG. 1 is a flowchart of a process for iteratively presenting a userwith a graphical user interface (GUI) for tagging image objectsidentified by image analysis of media content items in storageassociated with a certain user, according to some embodiments of thepresent invention; and

FIG. 2 is a schematic illustration of a system that comprises one ormore central server(s) 201 and one or more client modules each executedusing processor(s) of one or more client devices for monitoring userbehavior, according to some embodiments of the present invention.

DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

The present invention, in some embodiments thereof, relates to imageobject tagging and, more specifically, but not exclusively, to methodsand systems of tagging image objects in images of a specific user.

According to some embodiments of the present invention, there areprovided methods and systems for tagging media objects such as physicalelements or unhuman animals imaged in media content items of a user(e.g. image or video file) with user defined metadata, such as keywords,gathered in one of a plurality of suitable events using a designatedGUI. Optionally, a presentation of the designated GUI, for instance awindow of an application or a webpage presented by a browser, istriggered when a suitable event is detected by an analysis of outputs ofsensors of a client device, for instance an accelerometer or an imagesensor which are indicative of an activity of the user. The suitableevent may be detected by an analysis of outputs of applications executedon the client device.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the Examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network.

The computer readable program instructions may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). In some embodiments, electronic circuitry including, forexample, programmable logic circuitry, field-programmable gate arrays(FPGA), or programmable logic arrays (PLA) may execute the computerreadable program instructions by utilizing state information of thecomputer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Reference is now made to FIG. 1 which is a flowchart of a process 100for iteratively presenting a user with a graphical user interface (GUI)for tagging image objects identified by image analysis of media contentitems in storage associated with a certain user, according to someembodiments of the present invention. The process 100 allows iterativelygathering user-generated metadata describing the image objects from thesubjective point of view of the certain user in a minimally invasivemanner, for example during events identified as suitable for the user.As used herein, a media content item includes an image, a frame, avideo, and/or the like.

As further described below, the GUI, which is optionally presented on amobile device of the certain user at events identified as suitable forthe certain user, is presented in each of the iterations to gatheruser-generated metadata of a certain image object, for instance a userprovided name or description of the image object. The suitable push GUIevents are optionally identified based on behavior analysis of datagathered by a module executed on a device of the user, or instance amodule that gather data from input components of the device (e.g.accelerometer(s), image sensor(s), and a touch screen) and/or from ananalysis of processes such as applications executed on the device. Themodule is optionally a code stored in a program store of the clientdevice, for instance physical memory of the client device.

The gathered user-generated metadata is optionally used for respondingto media content search queries, for instance text based queries, of thecertain user, for instance a query for image search in a personalizedimage library, a query for a World Wide Web (WWW) image search, a queryfor a video search and/or used for personalizing user preferences.

Reference is also made to FIG. 2 which is a schematic illustration of asystem 200 that comprises one or more central server(s) 201 and one ormore client modules 206 each executed using processor(s) 211 of one ormore client devices 207 for monitoring user behavior, according to someembodiments of the present invention. A client device, such as 207, maybe any user carried device such as a Smartphone, tablet, laptop, and/ora wearable device such as a smartwatch and/or smart glasses. The clientdevice 207 optionally includes one or more image sensors such as camerasfor capturing images. The central server(s) 201, which optionallyincludes hardware processors 214 and central module 213, may be executedon hardware servers and/or virtual machine(s) using hardware resourcessuch as hardware processors and storage. FIG. 2 also depicts an imagedatabase 210, such as cloud image storage.

In use as shown at 101, a plurality of media content items are processedto identify one or more image objects (e.g. unhuman animals and/orphysical elements such as art pieces, room backgrounds, furniture, carsand/or the like). The plurality of media items may be processed locallyby execution of the client module 206 and/or in a central platform byexecution of module 213. The processed media objects may be documentedin media library(ies) of a certain user, for instance taken from astorage, such as items in one or more personal media libraries, imagesand/or video shared in a social media platform and/or media items asthey are being added to a storage, for instance after capturing and/orloading to a personal media library and/or uploading to a social mediaand/or a messaging platform. The image objects may be identified byexecuting a plurality of object recognition algorithms, for instanceFeature-based methods wherein feasible matches between object featuresand image features are found. An exemplary Feature-based method extractsfeatures from the objects to be recognized and the images to be searchedsuch as surface patches, corners, and linear edges. Another recognitionalgorithm is an interpretation trees method wherein feasible matchesbetween object features and image features are searched using a treewherein each node in the tree represents a set of matches (the root noderepresents empty set) and each other node is the union of the matches inthe parent node and one additional match.

Optionally, an image object dataset documenting a plurality of imageobjects is created and/or updated, for instance a table or a matrix or arecord set wherein each identified image object is mapped to an imageobject unique identifier (e.g. a number), one or more image objectclassifications (e.g. a car, an animal, a person, and a furniture, atree, a house, and/or any combination of image objects), automaticallyextracted image object metadata (e.g. color, brand, size, and/or thelike), image capturing or storing location, image capturing or storingdevice identifier, a capturing user identifier, automatically extractedimage recognition outputs (e.g. corners, surface patches, corners, andlinear edges) and/or reference(s) to one or more origin media contentelements, for instance a link storage address and/or a file name. Theimage object dataset maybe stored locally at the client device 207and/or at database 210 and/or any network based storage. The imageobject dataset also include fields for user provided metadata per imageobject. In use, each time a new media content item is captured and/oradded to a targeted media library, the image object dataset is updated.This automatically associates existing metadata with the new mediacontent image and/or with the new image objects. Optionally, a mediacontent item as a whole may be identified as an image object.

Optionally, as shown at 102, media content storage and/or capturingevents may be monitored to assure iterative analysis of media contentitems, for instance continuously updating the image object dataset by ananalysis of each new media content item for extraction of new mediaobject and/or updating of the image object dataset for documenting apresence of previously identified media objects in new media contentitems.

As shown at 103, suitable push GUI events for presenting the user with auser metadata acquiring graphical user interface (GUI) for adding usermetadata regarding one of the image objects are identified, for instanceby an analysis of monitored outputs or execution of sensors orapplications of the client device 207 and/or an analysis of a usagepattern of the user (how the user uses the client device 207).

A suitable push GUI event is optionally a timeframe during which themetadata acquiring GUI or an alert linked to the GUI, for instance anotification, is presented to the user. A suitable push GUI event may bedetected by an analysis of the output(s) of a monitoring process, forinstance a routine of the module 206 executed by the processor(s) 211.The analysis may be applied to detect a usage pattern of applicationsinstalled on the client device 207 by the user, for instance when theuser uses a business application such as an email client, when the useruses a leisure application such as a game, when the user does not userapplications, when the user uses a communication application such as anemail client, an instant messaging application, and/or a telephoneapplication. Based on this usage pattern a suitable push GUI event maybe identified, for instance just after (e.g. few seconds or minutes) theuser used a leisure application just after (e.g. few seconds or minutes)the user finished a communication using a communication applicationand/or or the like.

Optionally, the suitable push GUI events are time spaced, for instancebased on a predefined time spacing period (e.g. a GUI may be presentedno more than once a day, a week, or a month). Optionally, the suitablepush GUI events are synchronized according to an analysis of thecalendar, for instance on open slots only. Optionally, the suitable pushGUI events are synchronized according the time of the day or the day ofthe week.

Optionally, the suitable push GUI events are randomly timed, forinstance based on an output of a random generation module. Additionallyor alternatively, the suitable push GUI events are timed based on userinput, for instance in a low frequency, for instance once a week, in amedium frequency, for instance one 2 time a day, or in a high frequency,for instance every hour.

Additionally or alternatively, the suitable push GUI events are timedbased on a learning mechanism that learns the behavior of a user basedon responsiveness to previous suitable push GUI events. In suchembodiments, a time of location of a decline or a delay of apresentation of a GUI during a GUI event may be recorded, for instancefrom outputs of the timer and/or the locational data service of theclient terminal and used for identifying when and/or where the userresponsiveness to suitable push GUI events is low, high and/or medium.

Additionally or alternatively, usage patterns of a user during aplurality of days are analyzed using a statistical classier to estimatewhen the user is more likely to respond to a challenge presented to himby the metadata acquiring GUI.

Additionally or alternatively, one or more suggested tags are selectedfor the image object, for instance based on the classification of theimage object, optionally using a database of suggested tags perclassification. In such a manner, an identified car may provided withtags such as “my car”, “shagmobile”, “cool car”, and/or the like.

As shown at 104, in each suitable push GUI event a metadata acquiringGUI is presented. Optionally, in each suitable push GUI event one ormore media objects are selected, for instance based on a schedulingmythology such as first in first out, last in first out, most commonlyidentified media object, and/or any combination thereof.

The metadata acquiring GUI optionally presents the certain user with oneor more media content items, such as image(s) depicting an image objectand challenge the user to provide a user defined metadata such as atext, a tag, and/or a user indication. For instance, the user may inputa text describing or naming an image object and/or select one out of anumber of optional tags for an image object, for instance one out of anumber of options identified by application object recognition analysisprocesses. For example, the GUI is a window presenting image(s) orportion(s) of image(s) depicting an image object, a question selectedaccording to the classification of the image object and a field allowinga user to input text related to the image object. Optionally, a contouror other graphical element(s) marking the image object are used. Inanother example, a button allowing a user to record audibly adescription is presented to the user. The recorded content is thenconverted to text as known in the art.

Optionally, the GUI allows the certain user to request a presentation ofan additional image object for tagging.

Optionally, the GUI allows the certain user to request another user totag an image object, for instance by sending an image depicting theimage object to one or more of his friends in a social media servicesuch as Facebook™.

Optionally, the GUI allows the certain user to rank a media object item.The rank is documented, for instance in the media object dataset, andmay be used for ordering matching media content items which areretrieved in response to a search query.

As indicated above, user defined metadata is associated with an imageobject identified and tagged using recognition algorithms. In use, asfurther explained below, the user defined metadata may be used forautomatically tagging image objects in new media content items, such asnew images and video files. Such automatic tagging may require matchingbetween object(s) in the new media content items and the tagged mediaobjects or characteristics thereof, for instance surface patches,corners, and linear edges which are documented in a respective record.Such matching may be performed in different levels of certainty. Forinstance, a match may be ranked as a high probability match, a lowprobability match, and/or an average probability match.

In such embodiments, a media object identified as having a lowprobability match or an average probability match with a tagged mediaobject and/or identified as matching to a plurality of tagged mediaobjects may be verified by presenting, using the GUI, a clarificationquestion that uses data from the tagged media object(s). For instance, aGUI that presents an image object identified is a dog and matched with adog associated with the user defined metadata “Barney” and a dogassociated with the user defined metadata “Gringo” also present aselection question requiring form the user to indicate whether theimaged animal is “Barney” or “Gringo”. Additionally or alternatively,the GUI is used to validate automatic recognition of a media object by arecognition algorithm, for instance by presenting the user with aquestion is that dog is “Barney”.

The metadata acquiring GUI optionally provides the certain user with animage object marking tool, for instance a scribble marker that allowsthe user to select a sub-media object in a presented media content item,for instance a face, a person, a car, a furniture and/or any physicalelement found in a presented media object which may be even the mediacontent item as a whole. In such an embodiment, a new image object maybe identified and added to the image object dataset without any amarking action. The object delineation may be done using known imageprocessing functions.

Optionally, after the marking is completed, the user is required toprovide user defined metadata and/or to confirm an automaticclassification.

As shown at 105, the user defined metadata is used for tagging therelated image object, for instance added to the image object dataset,for instance in designated fields as described above.

As shown at 106, this process may be repeated until all recognized imageobjects are tagged with user defined metadata.

After the image object dataset is at least partly built and includesuser defined metadata tagging at least some of the identified imageobjects, the certain user may search, for instance keyword search, formedia content items based on a search query that include search terms heinputted as tags.

User defined data added to the image object dataset may be used forupdating the image object dataset, for instance by processing theplurality of media content items to identify the tagged image objectusing user defined metadata. In such embodiments, an image processingbased on the user defined metadata may be used, for instance analgorithm searching features defined by the user defined metadata.

User defined data may be a negative indication. For instance a negativeresponse to the question “is that Barney?” may be used for reducing thetag “Barney” and/or for adding a tag “not Barney”.

Additionally or alternatively, the search involves a natural languageprocessing (NLP) process wherein free text is analyzed is supported byusing user defined metadata. In such embodiment, the understanding ofthe user input may be supported by the media object dataset.

Additionally or alternatively, the media object dataset is used formanaging storage of media content items, for instance in the memory ofthe client terminal and/or in a cloud storage. The management may beperformed by archiving media content items tagged as junk, scored with alow score and/or depict certain media objects, for instance mediaobjects which have not been documented in new media content items for acertain period, media objects selected by the user, and/or predefinedmedia objects such as documents, screenshots, and/or the like. Thearchiving may be performed after a predefined period.

Additionally or alternatively, the media object dataset is used formanaging access level to media content items, for instance based on thepresence or absence of certain media objects therein. In such a manner,media content items depicting a certain person or element may be storedin a designated storage, password protected and/or present after abiometric identification of a user accessing a respective database.

Additionally or alternatively, the user defined metadata is used forpersonalizing general visual search of the user in general databases,for instance in a WWW search engine, a desktop search and/or any otheruser keyword search. In such embodiments the image object search may beused to convert user defined metadata into keywords describing therespective media content item.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

It is expected that during the life of a patent maturing from thisapplication many relevant methods and systems will be developed and thescope of the term a processor, a client device, an imaging sensor and anetwork is intended to include all such new technologies a priori.

As used herein the term “about” refers to ±10%.

The terms “comprises”, “comprising”, “includes”, “including”, “having”and their conjugates mean “including but not limited to”. This termencompasses the terms “consisting of” and “consisting essentially of”.

The phrase “consisting essentially of” means that the composition ormethod may include additional ingredients and/or steps, but only if theadditional ingredients and/or steps do not materially alter the basicand novel characteristics of the claimed composition or method.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a compound” or “at least one compound” may include a pluralityof compounds, including mixtures thereof.

The word “exemplary” is used herein to mean “serving as an example,instance or illustration”. Any embodiment described as “exemplary” isnot necessarily to be construed as preferred or advantageous over otherembodiments and/or to exclude the incorporation of features from otherembodiments.

The word “optionally” is used herein to mean “is provided in someembodiments and not provided in other embodiments”. Any particularembodiment of the invention may include a plurality of “optional”features unless such features conflict.

Throughout this application, various embodiments of this invention maybe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Accordingly, the description of a range should be consideredto have specifically disclosed all the possible subranges as well asindividual numerical values within that range. For example, descriptionof a range such as from 1 to 6 should be considered to have specificallydisclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numberswithin that range, for example, 1, 2, 3, 4, 5, and 6. This appliesregardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to includeany cited numeral (fractional or integral) within the indicated range.The phrases “ranging/ranges between” a first indicate number and asecond indicate number and “ranging/ranges from” a first indicate number“to” a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numerals therebetween.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

All publications, patents and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present invention. To the extent thatsection headings are used, they should not be construed as necessarilylimiting. In addition, any priority document(s) of this applicationis/are hereby incorporated herein by reference in its/their entirety.

What is claimed is:
 1. A method of controlling a presentation of agraphical user interface (GUI), comprising: identifying at least oneusage pattern according to an analysis of at least one of an executionof a plurality of applications and outputs of at least one sensor of aclient device; identifying a plurality of suitable push graphical userinterface (GUI) events during which the user is more likely to respondto a challenge presented to him according to the at least one usagepattern; and in each one of the plurality of suitable push GUI events:triggering a presentation of a question encouraging an input comprisinguser data responding the challenge, the presentation is performed usingthe GUI on a display of the client device.
 2. The method of claim 1,wherein the input is a textual input.
 3. The method of claim 1, furthercomprising: acquiring the user data from the input of the user using theGUI; and documenting the user data in association with the challenge. 4.The method of claim 1, wherein the at least one usage pattern isidentified according to an analysis of the execution of the plurality ofapplications.
 5. The method of claim 1, wherein the at least one usagepattern is identified according to an analysis of the outputs of atleast one sensor.
 6. The method of claim 1, wherein the plurality ofsuitable push GUI events are timed spaced from one another by a periodof at least one day.
 7. The method of claim 1, wherein the GUI is awindow presenting the question encouraging the input.
 8. The method ofclaim 1, further comprising receiving a search query comprising at leastone keyword matching to respective the user data from the user andretrieving a respective content in response to the search query.
 9. Themethod of claim 1, wherein the plurality of suitable push GUI events areidentified by an analysis of the output of an image sensor.
 10. Themethod of claim 1, wherein the at least one usage pattern is identifiedaccording to an analysis of a usage of a touch screen.
 11. The method ofclaim 1, wherein the at least one usage pattern is identified accordingto an analysis of the usage of type of the application.
 12. A clientdevice for controlling a presentation of a graphical user interface(GUI), comprising: a display; at least one sensor; a processor adaptedto execute a plurality of applications and a code for: identifying atleast one usage pattern according to an analysis of at least one of theexecution of the plurality of applications and outputs of the at leastone sensor; identifying a plurality of suitable push graphical userinterface (GUI) events during which the user is more likely to respondto a challenge presented to him according to the at least one usagepattern; in each one of the plurality of suitable push GUI events:triggering a presentation of a question encouraging an input comprisinguser data responding the challenge, the presentation is performed usingthe GUI on the display.